Skip to content

feat: add together to native OpenAI-compatible provider routing - #8804

Open
Felix Shi (shi-felix) wants to merge 3 commits into
crewAIInc:mainfrom
shi-felix:feat/together-native-provider
Open

Felix Shi (shi-felix) wants to merge 3 commits into
crewAIInc:mainfrom
shi-felix:feat/together-native-provider

Conversation

@shi-felix

@shi-felix Felix Shi (shi-felix) commented Oct 30, 2026 •

Copy link
Copy Markdown

Summary

together now routes through the native OpenAICompatibleCompletion instead of falling back to LiteLLM, so LLM(model="together/<organization>/<model-id>") works without the LiteLLM extra.

Fixes #8011.

Changes

  • lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py: add the together ProviderConfig (https://api.together.ai/v1, TOGETHER_API_KEY, optional TOGETHER_BASE_URL) and list it in the class docstring.
  • lib/crewai/src/crewai/llm.py: add together to SUPPORTED_NATIVE_PROVIDERS, PROVIDER_ALIASES, the native-provider set in _get_native_provider, and _matches_provider_pattern.
  • lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py: coverage for the registry entry, the missing-key error (and that an OpenAI key is not reused), prefixed and explicit-provider routing, TOGETHER_BASE_URL, explicit-config precedence, org/model ids, and the resolved client params.
  • docs/edge/{en,ar,ko,pt-BR}/concepts/llms.mdx: a Together AI accordion.

Only the together/ prefix is stripped, so together/Qwen/Qwen3-235B-A22B-Instruct-2507-tput reaches the API as Qwen/Qwen3-235B-A22B-Instruct-2507-tput. No model catalog is hard-coded.

is_litellm=True still forces the LiteLLM path, and the other native providers are untouched.

Testing

python -m pytest lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py -p no:randomly -q -n0 --basetemp=...
73 passed in 24.39s
python -m ruff check <changed files>          # All checks passed!
python -m ruff format --check <changed files> # 2 files already formatted

Python 3.13.14 on Windows. No provider credentials and no paid API calls were used.

AI assistance

Written with AI assistance (WorkBuddy) and reviewed before submission.


Note

Low Risk
Scoped additive change to LLM routing and OpenAI-compatible provider config, with tests and docs; existing providers and LiteLLM fallback behavior are unchanged unless is_litellm=True.

Overview
Adds Together AI as a first-class native provider so LLM(model="together/…") uses OpenAICompatibleCompletion instead of LiteLLM (no crewai[litellm] extra).

Configuration follows the other OpenAI-compatible hosts: default base https://api.together.ai/v1, required TOGETHER_API_KEY, optional TOGETHER_BASE_URL or explicit api_key/base_url. Auth is provider-specific—an OPENAI_API_KEY alone does not satisfy Together.

Model strings keep the Together organization/model path after routing; only the together/ prefix is removed (e.g. together/Qwen/… → Qwen/… at the API). Routing is wired through llm.py provider lists and pattern matching, with tests for registry, routing, env overrides, and client params.

Docs add a Together AI accordion in en, ar, ko, and pt-BR LLM guides, aligned with the DeepInfra section.

Reviewed by Cursor Bugbot for commit d9d8e07. Bugbot is set up for automated code reviews on this repo. Configure here.

@mintlify

mintlify Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit dfa0e6d.

Approve preview

@strix-security

strix-security Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Strix Security Review

Warning

This pull request has 117 commits after the last Strix review (e5b5d3c). Strix has not reviewed these changes.
Automatic review on push is off for this repository. To review the latest changes, tag @strix-security in a comment, or turn on re-review on push.

✅ No open security findings.

PR summary: Adds `together` as a native OpenAI-compatible provider so `LLM(model="together/<org>/<model>")` routes through `OpenAICompatibleCompletion` instead of falling back to LiteLLM, wired up via a new `ProviderConfig` entry and matching routing entries, with tests and localized docs.

Review summary

Reviewed the full diff: the new together ProviderConfig, the four together routing additions in llm.py, the four docs accordions, and the added tests. The provider is registered with TOGETHER_API_KEY only (no default_api_key, api_key_required=True), and the tests confirm an OpenAI key alone cannot satisfy it; routing is an exact-key map lookup, so provider="together" cannot resolve to another provider's config, and the new _matches_provider_pattern early return is only consulted once the provider is already together, so no other model's routing or credentials are affected. Prefix stripping removes only the first segment, so org/model ids reach the API intact. The default base URL is Together AI's own vendor-documented OpenAI-compatible endpoint (confirmed against the provider's current documentation and live TLS/error-identity checks), so the key is not directed at a third-party host, and the TOGETHER_BASE_URL override is documented with https and is operator-controlled exactly like the sibling providers. No security issues were confirmed in the changed code.

Ruled out (2)
  • Can a leftover global OPENAI_BASE_URL send TOGETHER_API_KEY to an unintended gateway host?
    Both the env-injection helper and the base-url precedence resolver are unchanged by this PR, the same behavior already existed for every sibling provider and for the pre-PR LiteLLM route, and base_url comes only from operator environment variables or explicit developer kwargs, never from an attacke…
  • Does the unvalidated TOGETHER_BASE_URL allow the API key to be sent over plaintext HTTP?
    `_resolve_base_url` is not part of the diff and behaves identically for all eight pre-existing providers, the variable is read solely from the operator's own process environment, and the PR's code default and every documented example use https.

Review 1 on this PR · e5b5d3c · requested Oct 30 05:45 UTC when the PR was opened


Reviewed by Strix
Re-run review · Configure security review settings

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add Together AI to native OpenAI-compatible provider routing

1 participant